{
 "cells": [
  {
   "cell_type": "raw",
   "id": "1c95cd76",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "---\n",
    "sidebar_label: IBM watsonx.ai\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70996d8a",
   "metadata": {},
   "source": [
    "# ChatWatsonx\n",
    "\n",
    ">ChatWatsonx is a wrapper for IBM [watsonx.ai](https://www.ibm.com/products/watsonx-ai) foundation models.\n",
    "\n",
    "The aim of these examples is to show how to communicate with `watsonx.ai` models using `LangChain` LLMs API."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef7b088a",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "### Integration details\n",
    "| Class | Package | Local | Serializable | [JS support](https://js.langchain.com/v0.2/docs/integrations/chat/openai) | Package downloads | Package latest |\n",
    "| :--- | :--- | :---: | :---: |  :---: | :---: | :---: |\n",
    "| [ChatWatsonx](https://api.python.langchain.com/en/latest/ibm_api_reference.html) | [langchain-ibm](https://api.python.langchain.com/en/latest/ibm_api_reference.html) | ❌ | ❌ | ❌ | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain-ibm?style=flat-square&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain-ibm?style=flat-square&label=%20) |\n",
    "\n",
    "### Model features\n",
    "| [Tool calling](/docs/how_to/tool_calling/) | [Structured output](/docs/how_to/structured_output/) | JSON mode | Image input | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | Native async | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
    "| :---: | :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n",
    "| ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f406e092",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "To access IBM watsonx.ai models you'll need to create an IBM watsonx.ai account, get an API key, and install the `langchain-ibm` integration package.\n",
    "\n",
    "### Credentials\n",
    "\n",
    "The cell below defines the credentials required to work with watsonx Foundation Model inferencing.\n",
    "\n",
    "**Action:** Provide the IBM Cloud user API key. For details, see\n",
    "[Managing user API keys](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "11d572a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "watsonx_api_key = getpass()\n",
    "os.environ[\"WATSONX_APIKEY\"] = watsonx_api_key"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c59782a7",
   "metadata": {},
   "source": [
    "Additionally you are able to pass additional secrets as an environment variable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f98c573c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"WATSONX_URL\"] = \"your service instance url\"\n",
    "os.environ[\"WATSONX_TOKEN\"] = \"your token for accessing the CPD cluster\"\n",
    "os.environ[\"WATSONX_PASSWORD\"] = \"your password for accessing the CPD cluster\"\n",
    "os.environ[\"WATSONX_USERNAME\"] = \"your username for accessing the CPD cluster\"\n",
    "os.environ[\"WATSONX_INSTANCE_ID\"] = \"your instance_id for accessing the CPD cluster\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3dc9176",
   "metadata": {},
   "source": [
    "### Installation\n",
    "\n",
    "The LangChain IBM integration lives in the `langchain-ibm` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "387eda86",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -qU langchain-ibm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e36acbef",
   "metadata": {},
   "source": [
    "## Instantiation\n",
    "\n",
    "You might need to adjust model `parameters` for different models or tasks. For details, refer to [Available MetaNames](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#metanames.GenTextParamsMetaNames)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "407cd500",
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    \"decoding_method\": \"sample\",\n",
    "    \"max_new_tokens\": 100,\n",
    "    \"min_new_tokens\": 1,\n",
    "    \"stop_sequences\": [\".\"],\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b586538",
   "metadata": {},
   "source": [
    "Initialize the `WatsonxLLM` class with the previously set parameters.\n",
    "\n",
    "\n",
    "**Note**: \n",
    "\n",
    "- To provide context for the API call, you must pass the `project_id` or `space_id`. To get your project or space ID, open your project or space, go to the **Manage** tab, and click **General**. For more information see: [Project documentation](https://www.ibm.com/docs/en/watsonx-as-a-service?topic=projects) or [Deployment space documentation](https://www.ibm.com/docs/en/watsonx/saas?topic=spaces-creating-deployment).\n",
    "- Depending on the region of your provisioned service instance, use one of the urls listed in [watsonx.ai API Authentication](https://ibm.github.io/watsonx-ai-python-sdk/setup_cloud.html#authentication).\n",
    "\n",
    "In this example, we’ll use the `project_id` and Dallas URL.\n",
    "\n",
    "\n",
    "You need to specify the `model_id` that will be used for inferencing. You can find the list of all the available models in [Supported foundation models](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#ibm_watsonx_ai.foundation_models.utils.enums.ModelTypes)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98371396",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_ibm import ChatWatsonx\n",
    "\n",
    "chat = ChatWatsonx(\n",
    "    model_id=\"ibm/granite-13b-chat-v2\",\n",
    "    url=\"https://us-south.ml.cloud.ibm.com\",\n",
    "    project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
    "    params=parameters,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2202f4e0",
   "metadata": {},
   "source": [
    "Alternatively, you can use Cloud Pak for Data credentials. For details, see [watsonx.ai software setup](https://ibm.github.io/watsonx-ai-python-sdk/setup_cpd.html).  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "243ecccb",
   "metadata": {},
   "outputs": [],
   "source": [
    "chat = ChatWatsonx(\n",
    "    model_id=\"ibm/granite-13b-chat-v2\",\n",
    "    url=\"PASTE YOUR URL HERE\",\n",
    "    username=\"PASTE YOUR USERNAME HERE\",\n",
    "    password=\"PASTE YOUR PASSWORD HERE\",\n",
    "    instance_id=\"openshift\",\n",
    "    version=\"4.8\",\n",
    "    project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
    "    params=parameters,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96ed13d4",
   "metadata": {},
   "source": [
    "Instead of `model_id`, you can also pass the `deployment_id` of the previously tuned model. The entire model tuning workflow is described in [Working with TuneExperiment and PromptTuner](https://ibm.github.io/watsonx-ai-python-sdk/pt_working_with_class_and_prompt_tuner.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08e66c88",
   "metadata": {},
   "outputs": [],
   "source": [
    "chat = ChatWatsonx(\n",
    "    deployment_id=\"PASTE YOUR DEPLOYMENT_ID HERE\",\n",
    "    url=\"https://us-south.ml.cloud.ibm.com\",\n",
    "    project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
    "    params=parameters,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f571001d",
   "metadata": {},
   "source": [
    "## Invocation\n",
    "\n",
    "To obtain completions, you can call the model directly using a string prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "beea2b5b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content=\"Je t'aime pour écouter la Rock.\", response_metadata={'token_usage': {'generated_token_count': 12, 'input_token_count': 28}, 'model_name': 'ibm/granite-13b-chat-v2', 'system_fingerprint': '', 'finish_reason': 'stop_sequence'}, id='run-05b305ce-5401-4a10-b557-41a4b15c7f6f-0')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Invocation\n",
    "\n",
    "messages = [\n",
    "    (\"system\", \"You are a helpful assistant that translates English to French.\"),\n",
    "    (\n",
    "        \"human\",\n",
    "        \"I love you for listening to Rock.\",\n",
    "    ),\n",
    "]\n",
    "\n",
    "chat.invoke(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "8ab1a25a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='Sure, I can help you with that! Horses are large, powerful mammals that belong to the family Equidae.', response_metadata={'token_usage': {'generated_token_count': 24, 'input_token_count': 24}, 'model_name': 'ibm/granite-13b-chat-v2', 'system_fingerprint': '', 'finish_reason': 'stop_sequence'}, id='run-391776ff-3b38-4768-91e8-ff64177149e5-0')"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Invocation multiple chat\n",
    "from langchain_core.messages import (\n",
    "    HumanMessage,\n",
    "    SystemMessage,\n",
    ")\n",
    "\n",
    "system_message = SystemMessage(\n",
    "    content=\"You are a helpful assistant which telling short-info about provided topic.\"\n",
    ")\n",
    "human_message = HumanMessage(content=\"horse\")\n",
    "\n",
    "chat.invoke([system_message, human_message])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20e4b568",
   "metadata": {},
   "source": [
    "## Chaining\n",
    "Create `ChatPromptTemplate` objects which will be responsible for creating a random question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "dd919925",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "system = (\n",
    "    \"You are a helpful assistant that translates {input_language} to {output_language}.\"\n",
    ")\n",
    "human = \"{input}\"\n",
    "prompt = ChatPromptTemplate.from_messages([(\"system\", system), (\"human\", human)])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a013a53",
   "metadata": {},
   "source": [
    "Provide a inputs and run the chain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "68160377",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='Ich liebe Python.', response_metadata={'token_usage': {'generated_token_count': 5, 'input_token_count': 23}, 'model_name': 'ibm/granite-13b-chat-v2', 'system_fingerprint': '', 'finish_reason': 'stop_sequence'}, id='run-1b1ccf5d-0e33-46f2-a087-e2a136ba1fb7-0')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain = prompt | chat\n",
    "chain.invoke(\n",
    "    {\n",
    "        \"input_language\": \"English\",\n",
    "        \"output_language\": \"German\",\n",
    "        \"input\": \"I love Python\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2c9da33",
   "metadata": {},
   "source": [
    "## Streaming the Model output \n",
    "\n",
    "You can stream the model output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3f63166a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The moon is a natural satellite of the Earth, and it has been a source of fascination for humans for centuries."
     ]
    }
   ],
   "source": [
    "system_message = SystemMessage(\n",
    "    content=\"You are a helpful assistant which telling short-info about provided topic.\"\n",
    ")\n",
    "human_message = HumanMessage(content=\"moon\")\n",
    "\n",
    "for chunk in chat.stream([system_message, human_message]):\n",
    "    print(chunk.content, end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a7a2aa1",
   "metadata": {},
   "source": [
    "## Batch the Model output \n",
    "\n",
    "You can batch the model output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9e948729",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[AIMessage(content='Cats are domestic animals that belong to the Felidae family.', response_metadata={'token_usage': {'generated_token_count': 13, 'input_token_count': 24}, 'model_name': 'ibm/granite-13b-chat-v2', 'system_fingerprint': '', 'finish_reason': 'stop_sequence'}, id='run-71a8bd7a-a1aa-497b-9bdd-a4d6fe1d471a-0'),\n",
       " AIMessage(content='Dogs are domesticated mammals of the family Canidae, characterized by their adaptability to various environments and social structures.', response_metadata={'token_usage': {'generated_token_count': 24, 'input_token_count': 24}, 'model_name': 'ibm/granite-13b-chat-v2', 'system_fingerprint': '', 'finish_reason': 'stop_sequence'}, id='run-22b7a0cb-e44a-4b68-9921-872f82dcd82b-0')]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "message_1 = [\n",
    "    SystemMessage(\n",
    "        content=\"You are a helpful assistant which telling short-info about provided topic.\"\n",
    "    ),\n",
    "    HumanMessage(content=\"cat\"),\n",
    "]\n",
    "message_2 = [\n",
    "    SystemMessage(\n",
    "        content=\"You are a helpful assistant which telling short-info about provided topic.\"\n",
    "    ),\n",
    "    HumanMessage(content=\"dog\"),\n",
    "]\n",
    "\n",
    "chat.batch([message_1, message_2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c739e1fe",
   "metadata": {},
   "source": [
    "## Tool calling\n",
    "\n",
    "### ChatWatsonx.bind_tools()\n",
    "\n",
    "Please note that `ChatWatsonx.bind_tools` is on beta state, so right now we only support `mistralai/mixtral-8x7b-instruct-v01` model.\n",
    "\n",
    "You should also redefine `max_new_tokens` parameter to get the entire model response. By default `max_new_tokens` is set to 20."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "328fce76",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_ibm import ChatWatsonx\n",
    "\n",
    "parameters = {\"max_new_tokens\": 200}\n",
    "\n",
    "chat = ChatWatsonx(\n",
    "    model_id=\"mistralai/mixtral-8x7b-instruct-v01\",\n",
    "    url=\"https://us-south.ml.cloud.ibm.com\",\n",
    "    project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
    "    params=parameters,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e1633a73",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.pydantic_v1 import BaseModel, Field\n",
    "\n",
    "\n",
    "class GetWeather(BaseModel):\n",
    "    \"\"\"Get the current weather in a given location\"\"\"\n",
    "\n",
    "    location: str = Field(..., description=\"The city and state, e.g. San Francisco, CA\")\n",
    "\n",
    "\n",
    "llm_with_tools = chat.bind_tools([GetWeather])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3bf9b8ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='', additional_kwargs={'function_call': {'type': 'function'}, 'tool_calls': [{'type': 'function', 'function': {'name': 'GetWeather', 'arguments': '{\"location\": \"Los Angeles\"}'}, 'id': None}, {'type': 'function', 'function': {'name': 'GetWeather', 'arguments': '{\"location\": \"New York\"}'}, 'id': None}]}, response_metadata={'token_usage': {'generated_token_count': 99, 'input_token_count': 320}, 'model_name': 'mistralai/mixtral-8x7b-instruct-v01', 'system_fingerprint': '', 'finish_reason': 'eos_token'}, id='run-38627104-f2ac-4edb-8390-d5425fb65979-0', tool_calls=[{'name': 'GetWeather', 'args': {'location': 'Los Angeles'}, 'id': None}, {'name': 'GetWeather', 'args': {'location': 'New York'}, 'id': None}])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ai_msg = llm_with_tools.invoke(\n",
    "    \"Which city is hotter today: LA or NY?\",\n",
    ")\n",
    "ai_msg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba03dbf4",
   "metadata": {},
   "source": [
    "### AIMessage.tool_calls\n",
    "Notice that the AIMessage has a `tool_calls` attribute. This contains in a standardized ToolCall format that is model-provider agnostic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "38f10ba7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'name': 'GetWeather', 'args': {'location': 'Los Angeles'}, 'id': None},\n",
       " {'name': 'GetWeather', 'args': {'location': 'New York'}, 'id': None}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ai_msg.tool_calls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ee72a59",
   "metadata": {},
   "source": [
    "## API reference\n",
    "\n",
    "For detailed documentation of all IBM watsonx.ai features and configurations head to the API reference: https://api.python.langchain.com/en/latest/ibm_api_reference.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.1.undefined"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
